Machine vision has successfully been integrated into many manufacturing systems for automated inspection and automated control. Video cameras capture images with adequate resolution, typically 512.times.512. The image data can be stored and processed by a computer. With a narrow field of view, the resolution is sufficient for examining small details or detecting flaws for quality control. However, inspection of large areas at high resolution requires handling a very large volume of data. The penalty for detailed automated inspection is the time necessary for image acquisition and for analysis of the data. Many automated inspection applications have not been economically feasible due to time constraints and computing costs.
Automated inspection at high resolution is quite common. However, systems proposed are too slow to examine large areas or numerous sites effectively. For example, U.S. Pat. No. 5,517,235 issued to Wasserman, uses video cameras with zoom lenses to vary the field of view for inspecting printed circuit boards. A smaller field of view, and hence higher resolution, is needed to examine boards or parts of boards with greater density of components. The inspection head is advanced across a viewing table to acquire a complete image. Multiple passes may be made to increase the resolution for examining critical areas flagged in the initial inspection. Physical movement of the head back and forth, and zoom lens control are relatively slow and not selective for examining specific locations. Even if only a small field of high resolution data is to be acquired, the head must pass over the complete object to locate the area.
Closer examination of specific sites of interest is also proposed as in U.S. Pat. No. 5,051,825 issued to Cochran and Austin, which uses a first video camera to determine the position and orientation of the article using geometrical determination, and a second video camera which simultaneously captures an image of higher resolution while the article is advanced across the field of view. This is a slow process. Acquiring a full image at high resolution presents greater time and system capability demands.
U.S. Pat. No. 4,872,052 issued to Liudzius, et al., discloses a system for examining semi-conductors which assembles images from multiple angles of view for examining a three-dimensional object at low resolution. The system also includes a high resolution camera which receives location information from the low resolution image to adjust the image registration to the standard comparison data. The high resolution camera is moveable on an X-Y plane parallel to that of the semi-conductor in order to acquire target image data. It is provided with a focus compensating device for fine adjustment to the high magnification without changing the focal length. Moving the camera across the object and further manipulating a focus compensating device are both complicated mechanically and slow. It is not anticipated by this prior art design to examine numerous targets at high resolution.
Slow mechanical positioning or acquisition of large volumes of unnecessary data have rendered prior art automated inspection systems too slow to be appropriate for high resolution examination of numerous sites, or for relatively high speed automated processing.
Scanners for precise and rapid positioning of optic mirrors have been used for raster scanning to acquire detailed composite information, such as in confocal microscopes, CAT SCAN and MRI medical imaging equipment, and in industry for example in three dimensional imaging techniques. The raster scan makes a complete pass across the object assembling data line by line. A more complex scanner system includes two axes of rotation, offering precise positioning to any point across a plane within angular limitations. Dual axis scanners have been used for directing lasers, for instance in targeting systems, and light show demonstrations. A dual axis scanner generally combines a high quality optic mirror mounted on a magnetically driven armature with a position detector and a digital driver. The accuracy of dual axis scanners can be within a few microradians.
Use of a two axis scanner is proposed in U.S. Pat. No. 5,608,563 issued to Matsumura et al. in order to rapidly position a scanner to lines which contain data only. The scanner is used in cooperation with a projection apparatus rather than for data acquisition. Time is saved by not scanning lines which do not contain data. However, the location of lines containing data is known in advance and stored in the system before scanning begins. The system cannot be used to locate previously unknown targets.
In the inspection of regular objects a comparison to a reference standard or template is easily made. Regular objects are well suited for geometrical analysis and standard comparison. Once position registration is established, target location information can be pre-programmed. The elements of interest have known configurations and positions, reducing processing time and scanning requirements.
Objects, such as food and textile products in particular, can be highly variable in shape and texture and pose a significantly more difficult problem for machine vision inspection. In some cases parameters can be prescribed, or statistical analysis can predict reliable results. Foreign objects, however, defy any classification of this kind. Foreign objects may have unpredictable size, shape, material, position, orientation, and number which require a more adaptable approach to automated detection and analysis leading to identification. The more complex processing and resultant time required make rapid detection and acquisition of targets image data more important for achieving an efficient system.
In both cases the rapid detection of targets, acquisition of data and reduced processing time are important in producing an effective automated system.
Human visual inspection is commonly used for quality control for rapidly identifying superficial imperfections or blemishes. The eye is quickly drawn to irregularities from a broad scan of the object. The response is referred to as an attention mechanism. Features, such as motion and contrast, primarily detected by human vision, are of interest to quantify in order to emulate the attention mechanism in automated systems. Other features such as color, texture and alignment can be used similarly by automated systems.
The present invention was developed for the inspection of fish fillets for defects, primarily parasites. This is an area which has been difficult to automate. Most production functions in fin fish processing have been fully automated, but defect inspection still requires skilled quality inspection personnel. Surface and near surface parasites are currently detected by human visual examination on a candling table which provides back lighting through the fish fillet and highlights characteristic shadows that correspond to parasites. This is, obviously, a labor-intensive operation which adds to the production costs.
This application presents significant additional challenges to automated inspection. The fillets do not have a regular shape or size. Objects of interest must be located and distinguished from other visual features such as blood spots, traces of skin or strands of stomach lining which often have quite similar characteristics. Unlike other industrial inspection applications, the parasites do not have a fixed shape, size or number and are often transparent offering very low contrast. To make determinations of this kind, relatively high resolution is required which can match human capabilities. To be commercially useful, analysis must occur in a relatively short time scale. Human analysis requires approximately 3 seconds per side of each fillet. An automated system with a throughput of one fillet per second is desirable for economic substitution.
The limitation to providing high resolution imaging at human equivalent or even closer detail is the shear volume of data such an operation would create. A high resolution image of a whole fillet would be too large for processing and analysis in a useful time scale. A fillet of an estimated 200 mm.times.400 mm, at human equivalent resolution of 5 lines per mm, would generate an array of approximately 3000.times.6000 pixels. The sensors necessary to pick up such a large array are very expensive. A system capable of capturing and processing data at the necessary rate cannot be economically implemented. Emulation of human eye performance cannot be achieved using brute force.
To achieve human equivalent resolution, it has been proposed in the present invention to mimic human visual processing. The human eye has a small central color sensitive area made up of cones capable of very high resolution. The majority of visual sensors which comprise the peripheral vision system are lower resolution monochrome rods. In human visual inspection, a broad scan by the peripheral vision system uses the attention mechanism to provide an initial identification of anomalies. The central vision system is then used to examine anomalies in detail for final identification. Eye movement, known as saccadic fixation, jumps to relocate the eye position approximately 3-10 times per second having the effect of assembling a composite view of successive high resolution images. Imitating this two stage vision hierarchy involves collection of low resolution scene information over a wide area (typically 0.6 radian solid angle), providing an attention mechanism and directing a narrow field of view (about 17 mrads at a resolution of about 0.1 mrad) for close inspection of features of interest to assemble substantially complete high resolution information. Human eye saccadic fixation rate at about 10 Hz has been emulated in a camera orienting system known as Agile Eye.TM.. However, the selective acquisition of high resolution information at relatively high speed (30-60 Hz) has not previously been achieved.
The present invention utilizing the post-objective scanning design to provide saccadic emulation, imitates human visual acquisition at an order of magnitude faster. By achieving a 30-60 Hz acquisition rate, data acquisition can be matched to the normal frame rate of a video camera, as commonly available, to capture 30 frames or 60 fields per second. Thus a relatively simple and inexpensive system is optimized to view an entirely different scene with each frame or field.